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ABSTRACT 

This study investigated the relationship between 
aptitude, as measured by Def^-nse Language Aptitude Battery (DLA3) 
scores, and oral proficiency as measured by the oral Proficiency 
Interview (OPI) scores of 72 Federal Bureau of Investigation (FBI) 
Special Agents who completed basic foreign language training at the 
Defense Language Institute (DLI). The hypothesis that the DLAB is a 
predictor of proficiency levels resulting from DLI training was not 
strongly sustained, as 40% of the Special Agents fell below the level 
2 training goal, and because many of the Special Agents who received 
low OPI scores received the highest aptitude scores. Self-assessment 
data on their language learning success were collected from 76 former 
students, and the following factors were identified as most 
influential to success: (1) motivation/perseverance; (2) instructor; 
and (3) personal ability/aptitude. Factors having a negative 
influence on language learning success included: (1) poor quality of 
materials; and (2) inappropriate grammar/vocabulary. (DJD) 
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The information in this paper formed the foundation for the FBI's portion of a Pane! 
Presentation: The Uses and Limitations or Aptitude Testing in Various Agencies, at 
the Interagency Languare Roundtable Invitational Symposium or Language 
Aptitude Testing. 
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Foreword 



The FBI Honors Intern Program stat ted in 1985. Every year. Honors Interns have 
been assigned to the Language Services Unit (LSD), Laboratory Division. In 1988, 
two Honors Interns were assigned to LSD. Miss Margaret Williams, a senior at Smith 
College, wasone of the two. The following paper was originally conceived as a 
research project to be carried out h Miss Williams, under my direction. Her task 
was to collect and compile all the data and to wru several papers about different 
aspects of related research The collecting of the dc^a was a painstaking and 
frustrating task. The bulk of the data relating to thr students and the instructional 
programs was nrovided by LSD's Trainir.g Program Manager, Elizabeth Porcell. The 
Tra'.ning Program has only been computerized a little over xwo years; thus, data for 
the total time span of the r .earch, 1981-1988, had co come from personnel files 
and other hard copies of materials kept in LSU files. DLAB data have never been 
computerized, and the hard copy files go back twenty years or more, often 
containing unclear and incomplete information of aptitude testing instruments, 
scores and dates. In spite of these obstacles. Miss Williams was able to find all 
available data. 

Miss Williams and I traveled to the Defense Language Institute (DLI) and met with 
Dr. John Lett, Victor Shaw and other DLI personnel in the area of testing. Computer 
printouts received from Victor Shaw literally arrived on the last day of Miss Williams' 
internship. She was able to write a rough draft primer for the present paper. 

Throughout the summer. Miss Williams closely collaborated with Contract Linguist 
Olga Navarrete. Mrs. Navarrete's experience as a trained oral proficiency tester and 
as a former foreign-language teacher allowed herto provide Miss Williams with the 
day-to-day linguistic and statistical expertise needed for *his project. Mrs. 
Navarrete conceived and designed most of the graphical representations contained 
in this paper. She calculated and double-checked all the figures contained herein, in 
addition to coauthoring many of the portions of the final paper with me. 

Ourthanksgo to Mr. Eugen'^^ Nakada, who did all the computer graphics, Mrs. 
Margaret Gulotta, LSU's Field Operations Program Manager, and to personnel at 
DLI for their assistance. I would also like to thank Mr. Kent E'sickfor his support as 
my greatest critic and his valuable assistance as a meticulous proofreader. 

In conclusion, I would like to say that this research paper would not have come 
about without the recognition by the Unit Chief of the Language Services Unit, SSA 
Bryce Christensen, of the necessity and importance of a study of this subject. 



Marijke Walker 
Testing Program Manager 
Language Services Unit 
Laboratory Division 
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APTITUDE AND LANGUAGE LEARNING OF FB! SPECIAL AGENTS 

For many years, the FBI has used Defense Language Aptitude Battery (DLAB) scores 
as one of the factors in the selection of FBI Special Agents (SAs) for basic foreign- 
language training a+the Defense Language Institute (DLI). This isdueto tne fact 
that DLI uses this instrument in its selection of military personnel for foreign- 
language training at this facility. DLI, the developer of the DLAB, has conducted 
validation studies which indicate that the DLAB is a valid predictor of success in the 
acquisition of a foreign language. In the DLI publication "The Defense Language 
Aptitude Battery" J976), Peterson and Al-Haik clearly demonstrated the high 
correlation between DLAB scores and final scores at the end of training. The DLAB 
was found to be an extremely good predictor of success in Russian, but the 
researcher<;also found that the DLAB's overall effectiveness as a predictor in all 
languages "produced consistently higher validities than ether predictors." (p. 3) 
Other researchers have fo ^nd that "aptitude test scores are reliable predictors of 
success by individuals (adolescents and adults) in learning a second language." 
(Child and Parry, "Preliminary Investigation of the Relationship between VORD, 
MLAT1 and Language Proficiency.") The latter study revealed "significant mild 
correlations between performance on the MLAT (composite scores) and speaking 
proficiency ...." (p. 16) Similar correlations were found regarding VORD. Charles 
Stansfield's correlation study of the Pimsleur Language Aptitude Battery and end- 
of-course achievement tests found a median correlation of .54. (Stansfield, 
"Pimsleur Language Aptitude Battery/' p. 7.) 

The aim of our research is to show the relationship, if any, between aptitude as 
measured by DLAB scores and oral proficiency as measured by the Oral Proficiency 
Interview (OPI) scores of FBI Special Agents who completed basic foreign-language 
training at DLI. Our primary research question and two secondary questions were: 
1) Is the DLAB a predictor of success in foreign-language learning as measured by 
OPI scores? 2) What other variables affect these outcomes as well? 3) What does 
the student feel contributes to his success? Our starting hypothesis was that there is 
a moderate correlation betwec?n the DLAB and oral-proficiency outcomes. Our 
second hypothesis merely stated that other variables do play a role in the overall 
learning process, but no attempt was made to measure the relative weight of such 
variables. And finally, we wanted to query the students for opinions as to what 
contributed to their successful language learning. 
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TheDLAB was introduced in 1977. It was developed underthe direction of Dr. 
Antoine R. Al-Haik, Chief of Test Development and Validation at DLI, for the 
purpose of screening military personnel for foreign-language training. The 
artificial language of the DLAB is loosely based on English and tests both inductive 
and deductive learning ability. The 126-item test isdivided into four parts: Parti is 
a b ographical inventory, Part II tests recognition of stress patterns, Part III tests for 
foreign-language grammar. It isdivided into four sections: Section I deals with 
nouns and adjectives, Section II tests possessive forms, Section III deals with sentence 
structure, and Section IV requires the use of a combination of the rules introduced 
in th*^ previous three sections. Part IV, Foreign Language Concept '-ormation, 
requires the examinee to form language concepts from pictures and identify the 
text which correctly conveys whal -»ees m the picture by genercilizing from the 
information acquired from the top of the page. (For a complete description, read 
The Defense Language Aptitude Battery (1976) .) Raw scores are converted to 
standard scores, and cutting scores vary for each language category. The FBI has 
used this instrument, with the same cutcing scores a** DLL 

Design and Method 

Until the present study, no research had been done to quantify the results of testing 
and training in the FBI, and records had been kept rather haphazardly. Our original 
sample consisted of 106 subjects, all Special Agents, male and female, ranging in 
age from 25 to 50, who attended basic foreign-language courses at DLI between 
1981and 1988. Only the 72 SAs, all current FBI employees, with known DLAB and 
OPI scores were used for this study. They were divided into groups according to the 
DLI division of language categories: I (Italian Jtalian-Sicilian) - 19 subjects; III 
(Polish, Vietnamese, Russian, Czech) - 26 subjects; and, IV (Arabic, Korean, and 
Chinese-Mandarin) - 27 subjects. There were not enough subjects in Category II. 
Students in Category I attended DLI for 36 weeks, and students in Categories III and 
IV attended for 47 weeks. 

The collection of data was a slow and difficult process. Information was 
incomplete, unclear and fragmented, probably d ue to the fact that aptitude tests 
were administered by different divisions in the Bureau, until the Language Services 
Unit (LSU) assumed this duty in 1986. Only clearly identified DLAB test data have 



been used for the purpose of this study. Five SAs had taken the DLAB twice, and the 
second score was used. OPI score data were incomplete as welL Sorne graduates 
from the basic course were never tested by the FBI, -^r they were tested after a 
considerable time period, which may have impacted on proficiency Hence, only OPI 
scores obtained soon after completion of basic training were used for this study. !n 
some instances, when no FBI OPI scores were available, OPI scores from DLI were 
used. Due to the problems in data collection, as well as other resource constraints, 
we keptthe construct simple by expressing most of the quantified information in 
percentages. 

Findings 

The mean, mode and median of the DLAB and the OPI scores obtained by SAs were 
distributed as follows: 



Figure 1 


DLAB 


Mean 


Mode 


Median 


Category 1 


108 


102 


105 


Category III 


122 


132 


123 


Category IV 


121 


107 


117 


Categories 1, 111,1V 


116 


132 


118 


0LA8 Scores 
Figure 2 


OPI 


Mean 


Mode 


Median 


Category 1 


2 


2 + 
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Category III 


2 


1 + 
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Category IV 


1 + 
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Categories 1, 111,1V 


2 


2/2 + 


2 


OPI Scores 
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In the process of the investigation, we found that the OPI scores in the combined 
three language categories ranged from 0 + to 3. Th^ distribution of scores showed 
some interesting patterns. 

Category I : The greatest number (7 out of 19), or 37%, scored a 2 + . Twenty-six 
percent scored a 2. Only one subject scored a 3 (representing 5% of the group). 
Category III : Nine out of 26 (35%), or the mode/'^ll in the I + range. The one 
0 + score in this group was the lowest OPI score in t'le entire study. 
Category IV : The mode fo. this category was level 2 (8 out of 27, or 30%). In this 
group no student achieved a level 3. 

From the combined total, only 10% of the 72 Agents achieved minimum professional 
proficiency -a 3-. Fifty percent, comprising both the median and the mode, 
received a 2 or a 2 + (25% each). (DLI's goal is to train to level 2.) Hence, forty 
percent fell below the desired goal. (During the combined years of '85, '86, and '87, 
32% of DLI graduates in the combined Cat'/gories I, III, and IV, mostly enlisted 
personnel in their late teens or early twenties, with no college education, scored 
the desired 2 or above on the OPI.) 
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The hypothesis that the DLAB is a predicto; of proficiency outcomes resulting from 
DLI training was not strongly sustained, as shown by the fluctuating correlations 
between the DLAB scores and the OPI outcomes. In some cases, candidates with the 
highest aptitude scores received low OPI scores, while some who had barely 
surpassed the ^Jtting score performed extremely well. 



--Figure 4. Scatterplot Categories I, III, IV (below)-- 
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A moderate correlation between high DLAB scores and high OP! scores is only 
demonstrated in the graphical representation of Language Category III. 



-Figu'-e 5. Scatterplot Category III (below)- 



There was no discernible positive correlation between DLAB and oral-proficiency 
scores in Categories I and IV. 
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Other Variables 



The FBI bases selection of SAs for basic training on a variety of factors, besides the 
DLAB. The Agent's initial application for language training must be voluntary. The 
Agent must have worked in the Bureau for at least two years, indicating that he has 
experience in investigative work. He must have Fully Successful performance 
ratings, or better, along with authorization for training from his Special Agent in 
Charge or Assistant Special Agent in Charge. The FBI also takes into account the 
Agent's ability to accept a transfer after training; the Agent must show that the 

desire to attend DLI is not actually an avoidance of some other undesired field 

\ 

office. ^ 

In addition to the above, the FBI gives priority to an Agent who has had past 
language-learning experience, which may include formal foreign language 
instruction in high school or college or in-country learning, or preferably both. In 
addition, an Agent's college grade-point average is given consideration, since it may 
be indicative of his ability and motivation in past academic pursuits, as may be 
advanced degrees. 

Findings Concerning Variables 

Regarding the secondary research question concerning variables other than 
aptitude having a bearing on the language-learning process, this study has only 
identified the selection criteria currently used to identify SAs for basic training, and 
no attempt has been made to measure their relative direct or indirect contril 'Jtion 
to the overall successful acquisition of a foreign language, or tiow these factors 
may affect proficiency outcomes. Other possible variables such as age, gender, etc. 
are not considered in the selection process, nor do we know what role these factors 
may possibly play in foreign-language learning. 

The FBI Special Agent Language Learner 

Based on Carroll's "1971 Model of Foreign Language Learning" and John Lett's 
adaptation of this, the "1986 DLI Model of Foreign Language Learning," the 
researchers developed a questionnaire to get student observations concerning the 
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variables in these models, and what they felt contributed to or detracted from their 
progressin basic foreign-language training. (The questionnaire has been appended 
to this paper.) Questionnaires were i;ent to the 106 SAs originally identified as 
having received basic foreign-language training during the period of 1981 through 
1988; 76 responded. All responses were collated and quantified. The students 
identified other factors, besides the variables mentioned in the models, is having a 
bearing on the overall learning process. The data regarding the most influential 
learner factors collected from the FBI students strongly reflected the variance 
contribution of learner factors cited by Jakobovits based on a study by Carroll, 1965; 
Carroll and Sapon, 1959; Flaugher, 1967; Gardner and Lambert, 1969, Pimsleur, 
Sundland, and Mc Intyre, 1964. 



(Leon A. Jakobovits, Foreign Language Learning: A Psychc li nquistic Analysis of 
the Issues , p. 98.) 

Jakobovits does point out that the numbers in this table are only an approximation. 
He also points out that grade-point average includes both intelligence and 
perseverance and hence may contribute up to 50% overall, especially smce grade- 
point average also is part of ability/aptitude. 

The "1988 FBI Model of Foreign Language Learning," based on student response, 
can be represented as follows: 



Variable 



Percent of variance explained 



Aptitude 
Intelligence 

Perseverance or motivation 
Others 



33 
20 
33 
14 



-Figure 6. 1988 FBI Model of Foreign Language Learning (p. 9)- 
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Figure 6 



METHOD OF INSTRUCTION 

CURRICULUM 
COURSE MATERIALS 
INSTRUCTORS 
LANGUAGE LAB 

CLASS SIZE 





STUDENT 
LEARNING 




ABILITY 

APTITUDE 



TIME STUDYING 
OUTSIDE OF CLASS 



MOTIVATION 
PERSEVERANCE 



PREVIOUS LANGUAGE 
EXPERIENCE 



*Based on FBI students* responses (N76) 



1988 FBI Model of Foreign Language Learning* 



Findings Concerning Student Opinion 

When asked to id'' ntify the most influential factors contributing to their language 
learning success, 78% of the total number of students chose 
motivation/perseverance, followed by 47% listing the instructor as playing an 
important positive role in the overall learning process. Personal ability/aptitude 
was selected by only 39 percent of the students as a positive factor aiding in 
successful language learning. Factors having a negative bearing on successful 
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language learning were identified as well. The following graphic representation 
shows the students' perception of positive and negative factors affecting their 
language learning. 



-Figure 7. ^ ibined Student Perception of Positive and Negative Factors (p. 1 '^)- 

Category I students followed a similar pattern, listing personal 
motivation/perseverance as the most important positive factor having a bearmg on 
language learning (59%). Poorquality of materials was chosen as negatively 
affecting ^^nguage learning (47%). 

"Figure 8. Category I Positive and Negative Factors (p. 12)- 

Seventy-two percent of the students in Category III chose personal ability/aptitude 
as the greatest positive factor. Twenty-eight percent felt that inappropriate 
vocabulary/grammartaught in class negatively affected their language learning, 
and 28% cited poor quality of course m-'^terials as a negative factor. 

-Figure 9. Category III Positive and Negative Factors (p. 13)- 

In Category IV, 100% of the students listed personal motivation/perseverance and 
only 10% chose personal ability/aptitude as the influential factors leading to 
successful language learning. Negative factors followed a pattern similar to 
Category III. 

-Figure 10. Category IV Positive and Negative Factors (p. 14)- 
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Conclusion and Recommendation 
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This research attempted to answer the question to what extent DLAB scores predict 
oral-proficiency outcomes resulting from basic language training at DLL Selection 
criteria (in addition to the DLAB) provided by the Training Program Manager, and 
learner factors checked by the FBI students in the questionnaires have been 
identified as variables which possibly may have a bearing on the overall outcome of 
such training. The present construct was rather unsophisticated in terms of 
statistical procedures. Due to the less-than-ideal sample size in each language 
category and the problems in gathering the data, continued collection of exact 
data, with good retrieval possibilities, will facilitate a more exact study in the 
future. Such data should be automated in order to do a computer analysis for exact 
statistical information. Such information will assist in the selection process as well 
as in determining the effectiveness of DLI training. It is imperative that the FBI have 
the means of selecting the best possible students and placing them in the best 
possible foreign-language training programs. 

Due to the important role of other variables, the DLAB should be used with caution 
and should not be the main selection factor in determining which personnel should 
receive foreign-language training at DLI . In the findings graphically represented in 
figures 7-10, it appears that motivation plays an equal or greater role in the 
acquisition of a second language, and hence, it may be wise to have a motivational 
inventory/questionnaire incorporated into the selection process. Additional 
research needs to be carried out to determine the weight of other variables 
affecting the outcome of language training/learning, such as age, sex, learner and 
cognitive styles/strategies, etc. The results of such research may aid in selecting 
students most likely to attain desired proficiency levels in the best possible 
instructional program, and thus provide the FBI, in a cost-effective manner, with SAs 
with usable skills tocarry out the overall FBI foreign-language mandate. Further 
research needs to answer the questions of what constitutes aptitude, how it can be 
measured, and the overall role it plays in successful language learning. 
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INSTRUCTIONS: Please take a few moments to fill out this brief survey regarding 
your language training at DLI and relevant information. If you wish to add 
comments on any topic, there is space provided after each question and at the end 
of the survey. As soon as you have completed it, send it back in the enclosed 
envelope. Thank you for your time 

QUESTIONNAIRE 
(check or respond as appropriate) 

NAME_ LANGUAGE 

COURSE LEVEL AND DATES OF ATTENDANCE: 

BASIC INTERMEDIATE ADVANCED 

NATIVE LANGUAGE 



AGE SEX MARITAL STATUS: Single Married Divorced Widowed 

PREVIOUS LANGUAGE EXPERIENCE: 

Language Where learned Years learned 



LEVEL OF EDUCATION: 

Bachelor's Master's Other In what field? 



Defense Language Aptitude Battery (DLAB) score (if known) 

Oral Proficiency Test score (if applicable) 

Administered by FBI, DLI 



1. DO YOU THINK YOUR DLAB SCORE IS REFLECTIVE OF YOUR ABILITY TO LEARN 
LANGUAGES? 

Yes No Don't know 



2. THE PACE OFTHE COURSE WAS 

Just right Too slow Too fast 



3. RATETHE EMPHASIS ON THE FOLLOWING AREAS 

Just right Too much Too little 

Grammar 

Vocabulary 

Oral drills 

Culture 



ERIC 



20 



4 RATE THE AMOUNT OF PRACTICE IN THE FOLLOWING AREAS 

Just right Too much Too httle 

In Writing 

In Reading . 

In Speaking 

In Listening 



5 THE MOST DIFFICULT ASPECT(S) 

Grammar 

Vocabulary 

Speaking 



OF THE LANGUAGE WASAA/ERE 

Reading comprenension _ 
■ Listening 
Other: 



6. THE EASIEST ASPECT(S) OF THE LANGUAGE WASAA/ERE 

Grammar Reading comprehension 

Vocabulary Listening 

Speaking Other: 



7.a) HOW MANY HOURS OUTSIDE OF CLASS PER DAY DID YOU WORK ON 
LEARNING THE LANGUAGE? 

b) YOU FEE .THAT THIS WAS Sufficient Too litile Too much ^ 



8.a) THE AMOUNT OF ENGLISH USED BY THE INSTRUCTOR IN THE CLASSROOM WAS 

None 

Very little 

Only for difficult explanations 

Too much 

b) I FEELTHE USE OF ENGLISH WAS 

Beneficial 

Necessary 

Inappropriate 

Distracting 
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9. YOU WOULD DESCRIBE YOUR OVERALL PROGRESS AS 

Excellent Satisfactory Unsatisfactory 



10. THE FACTOR(S) YOU FEEL CONTRIBUTED MOST TO YOUR SUCCESS IS/ARE 

Personal ability/aptitude 

Personal motivation/perseverance 

The instructor 

Course materials 

Language lab 

Other (explain) 



11. THE FACTOR(S) YOU FEEL HINDERED YOUR SUCCESS 

Inappropriate vocabulary/grammar taught in class 

Poor instruction 

Poor quality of course materials 

Lack of previous background m studying a foreign language 

Uncertainty about future assignments 

Other demotivators (explain) ^ 



12. RATE THE OVERALL IMPRESSION YOU HAVE HAD WHILE LEARNING THIS 
LANGUAGE. 

5-Outstanding 

4-Above average 

3-Average 

2-Belowaverage 

1-Poor 

0 



13. PLEASE ADD ANY OTHER COMMENTS ON ANY ASPECT OF YOUR LANGUAGE 
TRAINING AT DLL 



